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1 .   INTRODUCTION 

The  Illinois  Pattern  Recognition  Computer,  Illiac  III,  is 
a  multiprocessor  computer.   To  provide  greater  flexibility  in  system 
architecture,  the  Illiac  III  computer  is  so  designed  that  it  can 
initially  operate  with  a  minimum  number  of  processors,  and  later 
expand  to  full  capacity  as  its  load  increases.   To  nut  it  in  another 
way,  the  Illiac  III  is  a  system  of  modular  processors,  each  of  which 
can  be  plugged  into  or  disconnected  from  the  system.   To  provide  this 
flexibility  is  the  responsibility  of  the  communication  net.   This 
paper  presents  design  specifications  and  a  model  of  a  communication 
net  for  this  purpose.   In  the  context  of  the  Illiac  III  system,  this 
communication  net  is  called  the  Exchange  Net. 

By  way  of  orientation,  the  Illiac  III  computer  system  can 
rapidly  analyze  and  dissect  two-dimensional  picture  data  by  the 
Pattern  Articulation  Unit  (PAU).   Input  images  to  the  computer  are 
digitized  by  eight  CRT  flying  spot  scanners:   two  for  70  mm  film,  two 
for  k6   mm  film,  two  for  35  mm  film,  and  two  for  microscope  slides. 
These  scanners  can  also  operate  as  film  cameras  and  thus  serve  as 
both  input  and  output  devices. 

The  modules  of  the  Illiac  III  computer  include  the  PAU 
mentioned  above.   Two  Input/ Output  Processors  (lOP)  are  attached  via 
Channel  Interface  Units  and  Control  Units  to  the  various  Input/Output 
devices,  and  direct  all  data  transfer  to  and  from  these  devices.   Four 
Taxicrinic  Processors  (TP)  act  as  the  control  units  of  the  Illiac  III. 


Their  principal  activities  are  the  manipulation,  search,  and  systemi- 
zation  of  abstract  graphs,  which  are  the  output  of  the  PAU.  Although 
a  TP  can  perform  a  number  of  simple  arithmetic  operations,  the  more 
complicated  arithmetic  computations  are  done  by  the  Arithmetic  Units 
(AU),  of  which  there  are  two  in  the  Illiac  III.  An  Interrupt  Unit 
(IU)  keeps  track  of  the  status  of  all  TP's  and  IOP's  and  forwards 
interrupt  commands  whenever  necessary.   In  addition  to  the  above, 
there  are  four  main  storage  groups  in  the  Illiac  III  system.  Each 
storage  group  consists  of  a  fast  core  module  (l6,38^4  double  words, 
700  ns),  a  slow  core  module  (65,536  double  words ^  3^s),  and  a  dic- 
tionary core  (read-mostly,  262,  lM+  double  words,  ^250  ns  reading 
t  ime ) . 

Since  all  modules  operate  in  parallel,  care  must  be  exercised 
that  conditions  such  as  two  TP's  or  IOP's  accessing  the  same  memory 
unit  simultaneously  do  not  occur.   Similarly  results  from  one  arithmetic 
unit  must  be  sent  to  the  correct  TP,  etc.   Thus,  besides  providing 
interconnections  among  the  modules  of  the  Illiac  III,  the  Exchange  Net 
must  also  be  able  to  supervise  communications  between  these  modules. 
Both  of  these  functions  are  to  be  accomplished  without  excessive 
duplication  of  hardware, 

Chapter  2  lists  the  requirements  imposed  on  the  Exchange  Net 

by  the  surrounding  modules.   Chapter  3  describes  the  overall  function 

and  operation  of  the  Exchange  Net,  and  the  specifications  set  up  at 

the  Exchange  Net-Module  interface.   Subblocks  in  the  Exchange  Net  are 

also  listed  here.   Chapter  h   gives  detailed  description  of  the  subblocks 

of  the  Exchange  Net.   A  summary  is  presented  in  Chapter  5- 
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2.   EXCHANGE  NET  REQUIREMENTS 

2 .1  General  Requirements 

The  modules  of  the  Illiac  III  computer  send  data  to  one 
another  one  word  at  a  time.   A  word  in  the  Illiac  III  is  defined  to 
be  four  bytes  each  consisting  of  eight  data  bits,  one  flag  bit,  and 
one  parity  check  bit.   Along  with  each  word,  control  signals  are 
also  sent  by  the  originating  module  so  that  the  receiving  module 
knows  exactly  what  to  do  with  the  incoming  data.   The  control  signals 
required  vary  according  to  the  source  and  destination,  as  different 
modules  perform  different  functions.   The  Exchange  Net  is  respon- 
sible for  providing  facilities  for  the  simultaneous  transmission  of 
the  one  word  of  data  and  the  control  information.   Since  some  of  the 
modules  need  never  communicate  with  certain  other  modules,  data 
paths  and  control  paths  between  these  are  not  necessary  and  are  not 
provided. 

2 .2  Taxicrinic  Processor  Requirements 

Since  the  TP's  are  the  control  units  of  the  Illiac  III, 
they  must  communicate  with  all  other  modules  of  the  system.   In 
particular,  they  can  talk  directly  to  the  memory  units,  the  AU's, 
the  PAU  and  IU.   But,  since  they  can  interrupt  the  IOP's  and  also  one 
another  through  the  medium  of  the  IU,  direct  paths  between  the  TP's 
and  the  IOP's  are  not  necessary.   After  the  TP's  have  secured  services 
from  the  other  modules,  they  require  that  the  results  be  sent  back  to 


them.   Thus  paths  leading  from  the  memory  units,  AU' s,  PAU,  and 
the  IU  back  to  the  TP's  are  also  required. 

A  TP  will  send  to  a  memory  unit  and  its  Memory  Control 
thi  following  control  signals: 

Memory  Request  (i  bit) 

Transfer  Information  In  (l  bit) 

Transfer  Information  Out  (l  bit) 

Synchronous/Asynchronous  Mode  (l  bit; 

Output  Left  Word  (l  bit) 

Output  Right  Word  (l  bit) 

Write  Left  Word  (l  bit) 

Write  Right  Word  (l  bit) 

Memory  Disconnect  (l  bit) 
The  TP  requires  in  return  from  the  memory  any  output  plus  one  bit 
of  Out  Information  Ready/ In  Information  Received,  and  one  bit  of 
Parity  Error. 

When  communicating  with  the  AU,  the  TP  sends  the  following 
control  information: 

AU  Request  (l  bit) 

In  Information  Ready/ Out  Information  Received  (l  bit) 

Number  Type  (2  bits) 

Instruction  Variant  (h   bits) 

AU  Disconnect  (l  bit) 
The  TP  requires  in  return  from  the  AU  the  partial  or  final  results 
and  the  following  control  information: 


Out  Information  Ready;  In  Informati  >n  Received  (l  bit), 

Multi-Cycle  Interrupt  (l  bit), 

Bogus  Result  (l  bit), 

Parity  Error  (1  bit) . 

When  a  IT  needs  the  service  of  an  AU,  it  has  no  nref  :>■■  nc 
as  to  which  AH  it  should  us<  .   If,  however,  it  is  doing  a  multi-cycle 
operation,  then  the  T.I?  wants  the  whole  sequence  of  data  sent  to  the 
same  AU.   The  Exchange  Net  is  the  only  portion  of  the  machine  that  is 
constantly  in  touch  with  the  AU's  and  knows  their  conditions.   There- 
fore, when  a  TP  needs  an  AU,  it  requests  the  Exchange  Net  to  choose 
among  the  two  AU's  one  that  is  in  a  condition  to  service  it.   In  the 
case  that  both  AU' s  are  doing  multi-cycle  operations  (each  for  one 
TP)  and  a  third  TP  wishes  to  use  an  AU,  this  TP  will  request  the 
Exchange  Net  to  interrupt  one  of  the  AU' s  and  clear  the  AU  for  its 
use . 

The  requirements  for  TP-PAU  and  TP-IU  communications  have 
not  been  defined  yet  as  of  the  time  of  this  writing.   But,  it  is 
conceivable  that  the  TP  needs,  when  talking  to  the  PAU,  the  following 
control  signals: 

PAU  Request  (l  bit) 

In  Information  Ready/ Out  Information  Received  (l  bit) 

PAU  Disconnect  (l  bit) 
and  perhaps  a  few  bits  of  Instruction  Variant.   The  actual  instruc- 
tions for  the  PAU  will  be  sent  in  the  data  bytes.   The  PAU  should 
send  the  TP  a  Parity  Error  signal  if  parity  error  has  occurred. 


It  is  also  conceivable  that  a  TP  would  want  to  send  the  IU 
the  following  control  signals: 

IU  Request  (l  bit) 

In  Information  Ready  (l  bit), 

IU  Disconnect  (l  bit), 
and  expects  the  IU  to  return  a  Parity  Error  signal  should  a  purity 
test  fails  and  an  IU  Calling  signal  if  some  TP  or  IOP  wants  to 
interrupt  while  it  is  busy. 

2 . 3   Input/ Output  Processor  Requirements 

The  duty  of  the  IOP's  is  to  direct  data  transfer  to  and  from 
the  Input/ Output  devices.   These  Input/ Output  processors  communicate 
with  the  memory  units  much  of  the  time.   The  IOP's  also  want  to  be 
able  to  interrupt  the  TP's  through  the  IU .   Therefore,  data  and  con- 
trol paths  leading  from  the  IOP's  to  the  memory  units  and  the  IU  are 
necessary.   Likewise,  return  paths  from  the  memory  and  IU  are  required. 
Control  signals  needed  by  the  IOP's  when  talking  to  the  memory  units 
and  the  IU  are  exactly  the  same  as  those  needed  by  the  TP's  communicating 
with  these  units. 

2 .k     Memory  Requirements 

Apart  from  the  control  signals  required  by  the  TP's  and 
IOP's,  each  memory  unit  sends  to  the  Exchange  Net  certain  information 
about  itself  to  facilitate  its  own  service.   It  sends  to  the  Exchange 
Net  a  Request  signal  whenever  it  wants  to  deliver  information  to  the 


TP  or  IOP  currently  using  its  Facilities.   Since  the  memory  unit  do  : 
not  know  and  cannot,  keep  trad    '  its  users,  the  Exchange  Net  must 

provide  this  servic<  so  that  data  from  the  memory  unit  will  be:  sent 
to  the  correct  modu]  .     The  memory  unit  also  tells  the  Exchange 
Net  whether  or  not  it  is  busy,  so  that  it  will  not  be  disturbed 
while  on  a  job.   In  addition,  it  sends  the  Exchange  Net  a  Memory 
Attached  signal  so  that  the  Exchange  Net  knows  that  the  memory  unit 
is  connected  to  the  computer  system  and  that  its  power  is  turned  on. 
The  memory  unit  also  provides  a  Malfunction  signal  when  a  malfunction 
occurs . 

2 . 5  Arithmetic  Unit  Requirements 

Each  AU  keeps  the  Exchange  Net  informed  of  its  current 
status.   It  lets  the  Exchange  Net  know  if  it  is  busy,  if  it  is  doing 
a  multi-cycle  operation,  has  a  malfunction,  or  wants  a  communication 
path.   When  its  power  is  turned  on,  it  sends  to  the  Exchange  Net  an 
AU  Attached  signal  so  that  the  Exchange  Net  knows  of  its  existance. 
Like  the  memory  units,  an  AU  cannot  keep  track  of  its  users,  so  the 
job  is  left  to  the  Exchange  Net  to  deliver  outputs  from  an  AU  to  the 
correct  TP. 

2 .6  Pattern  Articulation  Unit  Requirements 

The  PAU  obtains  its  input  data  from  the  memory  units 
through  a  TP  and  sends  the  results  back  to  the  TP.   Like  the  memory 
units  and  the  AU's,  the  PAU  depends  on  the  Exchange  Net  to  direct  its 
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output  results  to  the  appropriate  TP.   The  PAU  forwards  to  the 
Exchange  Net  one  bit  of  PAU  Busy,  one  bit  of  PAU  Malfunction,  one 
bit  of  PAU  Attached,  and  one  bit  of  Exchange  Request. 

2 .7   Interrupt  Unit  Requirements 

To  better  understand  the  requirements  imposed  on  the 
Exchange  Net  by  the  IU,  it  is  necessary  to  know  the  basic  operations 
of  the  IU.   The  interrupt  unit  keeps  track  of  the  status  of  the  four 
TP's  and  the  two  IOP's.   The  IU  knows  if  a  TP  or  IOP  is  busy  with  a 
job,  free,  or  executing  an  interrupt  command.   When  an  interrupt 
request  comes  in,  the  IU  checks  to  see  if  the  processor  (TP  or  IOP) 
to  be  interrupted  is  in  a  state  to  accept  the  interrupt.   If  so,  the 
IU  forwards  the  interrupt  immediately.   However,  if  the  processor  to 
be  interrupted  is  busy,  the  IU  signals  the  processor  of  the  interrupt 
by  transmitting  an  IU  Calling  signal.   At  its  earliest  opportunity, 
the  signalled  processor  will  break  its  sequence  of  operations  and 
clear  its  status  with  the  IU.   The  IU  then  forwards  the  interrupt 
command. 

It  is  required  that  the  Exchange  Net  provides  an  IU  Calling 
line  from  the  IU  to  each  TP  and  IOP.   Since  the  processor  to  be 
interrupted  may  be  receiving  or  expecting  to  receive  data  from  a 
memory  unit,  an  AU,  or  the  PAU,  it  is  of  upmost  importance  that  the 
Exchange  Net  furnish  communication  paths  for  the  IU  Calling  signals 
which  will  not  disturb  the  data,  or  be  mistaken  for  data.   The  IU 
sends  the  Exchange  Net  one  bit  of  IU  Malfunction,  one  bit  of  IU 
Attached,  one  bit  of  Exchange  Request,  but  no  Busy  signal. 
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3.   GENERAL  ORGANIZATION  OF  THE  EXCHANGE  NET 

3 . 1   The  Six-by-Six  Parallel  Access  Net 

If  we  examine  the  requirements  imposed  on  the  Exchange  Net 
by  the  various  modules  of  the  Illiac  III  computer,  we  will  finci  that 
we  can  separate  these  modules  into  two  groups:   one  group  consisting 
of  the  four  TP's  and  the  two  I OP' s,  and  a  second  group  consisting  of 
the  12  memory  units,  the  two  AU's,  the  PAU,  and  the  IU.   Members  of 
one  group  must  talk  directly  to  members  of  the  other  group,  while 
members  within  the  same  group  need  never  communicate  directly  with 
each  other.   The  first  group  (TP's  and  IOP's)  appears  superior  to  the 
second,  since  the  second  group  only  services  requests  generated  by 
the  first,  group  but  not  vice  versa. 

Since  the  modules  in  the  Illiac  III  are  fast,  it  is 
important  that  they  can  access  other  modules  quickly  and  not  waste 
time  waiting  for  a  data  transfer  path.   While  the  fully  connected 
switching  network  shown  in  Figure  1  will  provide  all  needed  com- 
munications with  minimum  delay,  it  is  at  the  same  time  very  costly. 
The  large  number  of  switching  junctions  calls  for  a  prohibitive  amount 
of  hardware,  and  the  big  fanout  from  each  horizontal  bar  also  implies 
redundant  logic  and  hardware.   Since  there  are  only  six  "superior" 
modules,  at  most  six  of  the  vertical  bars  will  be  utilized  at  any 
one  time.   Therefore,  it  is  conceivable  that  if  we  re-group  the  service 
modules  appropriately,  and  assume  good  programming  control,  a  six-by-six 
crossbar  will  be  almost  as  good  as  a  six-by-sixteen  crossbar. 
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Basically,  the  Exchange  Not  provides  a  two-way,  six-by-six 

parallel  access  traffic  net  between  six  processors  (the  IP's  and  lOP's) 
and  six  Local  Exchanges  (sub-control  units  of  the  Exchange  Net).   Each 
Local  Exchange  handler,  three  of  the  service  units  (see  Figure  2).   For 
easier  reference  we  shall  call  the  TP's  and  IOP' s  "Processors",  and 
the  memory  units,  the  AU's,  the  PAU,  and  th^  IU,  "Units".   The  pattern 
of  traffic  from  units  to  processors  is  completely  independent  of  the 
pattern  of  traffic  from  processors  to  units.   Thus  the  traffic  paths 
shown  in  Figure  2  can  be  viewed  as  a  two-level  network.   The  upper 
level  handles  traffic  from  the  processors  to  the  Local  Exchanges,  and 
from  the  Local  Exchanges  to  the  various  units.   The  lower  level 
handles  traffic  from  the  units  to  the  Local  Exchanges,  and  from  the 
Local  Exchanges  to  the  processors.   Switching  control  for  traffic 
flow  at  the  two  levels  are  independent  of  each  other.   The  horizontal 
bars  in  Figure  2  are  called  horizontal  buses,  the  processor-to-unit 
portions  are  called  forward  horizontal  buses,  and  the  unit-to-processor 
portions  are  called  return  horizontal  buses.   Similarly,  the  vertical 
bars  below  the  Local  Exchanges  are  referred  to  as  vertical  buses. 
Again  the  processor-to-Local  Exchange  portions  are  called  forward 
vertical  buses,  and  the  Local  Exchange-to-processor  portions  are 
called  return  vertical  buses.   For  each  direction  the  capacity  of  the 
path  linking  each  processor  or  unit  to  the  Exchange  Net  is  five  bytes, 
four  data  bytes  and  one  control  byte.   All  bytes  are  10  bits,  and  the 
links  between  any  processor  or  unit  and  the  Exchange  Net  are  provided 
by  10-line  cables. 
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3.2  Inbus  and  Qutbus 

Ea.ch  five  bytes  of  communication  from  a  processor  into  a 
unit  is  called  an  Inbus .   By  extension,  an  Inbus  goes  from  a  processor 
into  the  Exchange  Net,  and  from  the  Exchange  Net  into  a  unit  (see 
Figure  3) . 

Each  five  bytes  of  communication  out  of  a  unit  to  a  processor 
is  called  an  Qutbus .   By  extension,  an  Outbus  comes  out  of  a  unit  to 
the  Exchange  Net  and  out  of  the  Exchange  Net  to  a  processor. 

3 .3  Interface  Specifications 

In  order  that  signals  from  a  source  processor  or  unit  will 
be  interpreted  properly  by  the  Exchange  Net  and  by  the  destination 
unit  or  processor,  some  kind  of  interface  convention  is  necessary. 
It  has  been  decided  that  all  signals  will  be  "true"  in  negative  logic 
at  the  Exchange  Net  interface.  With  this  convention,  any  physical 
disconnection  of  a  processor  or  a  unit  will  cause  all  signals  from 
it  to  go  to  '0'  (open  circuit  =  logical  '0')- 

The  Exchange  Net  requires  that  a  processor  requesting 
communication  service  indicate  its  destination  by  sending  the  Exchange 
Net  an  appropriate  code  as  listed  in  Table  I.   This  code  is  sent  to 
the  Exchange  Net  on  the  Inbus  control  byte.   The  Destination  Code  (DC) 
is  sent  when  the  request  is  raised  and  should  last  long  enough  to 
set  one  group  of  flip-flops  in  the  Exchange  Net.   It  is  then  replaced 
by  the  control  signals  (e.g.  instruction  variant  (IV)  to  an  AU)  to  be 
sent  to  the  destination  unit. 
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FIGURE  3.  ILLUSTRATION  OF  INBUS  AND  OUTBUS 
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Unit 


Starting  Core  Address 
(starts  at  2Uth  digit) 


Destination 
Code  0123 


FC 

;o) 

FC 

:d 

FC 

[2) 

FC 

[3) 

sc  1 

:o) 

SC  ( 

:d 

SC  1 

[2) 

SC  ( 

:3) 

DC  ( 

'0) 

DC  ( 

:d 

DC  ( 

2) 

DC  ( 

'3) 

IU 

PAU 

(not 

-  used) 

0101100 

0101101 

0101110 

0101111 

01100 

01101 

OHIO 

01111 

100 

101 

110 

111 


0000 
0001 
0010 
0011 
0100 
0101 
0110 
0111 
1000 
1001 
1010 
1011 
1100 
1101 
1110 

1111 


Table   1.      Destination  Code   for  Units 
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Interface  specifications  for  Inbus  and  Outbus  data  bytes 
are  as  follows: 

Each  data  byte  contains  eight  data  bits  (lines  1-8),  one 
flag  bit  (line  9)>  and  one  parity  check  bit  (line  10).   Parity  is 
odd,  i.e.  the  number  of  l's  in  any  10  bit  configuration  is  always 
odd.   Parity  is  checked  at  the  destination  unit  or  processor  but  not 
at  the  Exchange  Net.   All  10  bits  are  used  in  the  control  byte,  and 
thus  parity  is  not  checked. 

Inbus  control  specifications  at  the  Processor-Exchange 
interface  are  as  follows: 

line      function 

1  Request 

2  In  Information  Ready/ Out  Information  Received 

3  Transfer  Information  Out 

h  Number  Type  (o)/ (Synchronous/Asynchronous  Mode) 

5  Number  Type  (l) 

6  DC  (0)/IV(0)/0utput  Left  Word 

7  DC  (l)/IV(l)/ Output  Right  Word 

8  DC  (2)/IV(2)/Write  Left  Word 

9  DC  (3)/IV(3)/Write  Right  Word 
10        Unit  Disconnect 

A  few  words  of  explanation  are  needed  here.  The  "Transfer 
Information  In"  signal  for  the  memory  is  the  same  as  "In  Information 
Ready"  and  thus  uses  line  2.  For  line  3,  "Transfer  Information  Out" 
has  significance  only  for  memory  access.  When  communicating  with  an 


16 


AU,  the  source  processor  must  keep  this  line  'u:,  since  the  Exchange 
Net  uses  this  line  to  send  an  AU  Interrupt  signal  to  the  arithmetic 
unit.   For  line  k,    the  signal  represents  Number  Tyno  (u)  when,  talking 
to  an  AU,  and  indicates  Synchronous/ Asynchronous  Mode  when  talking 
to  the  memory.   Line  5  is  interpreted  as  Number  Type  (l)  by  the  AU, 
and  is  not  examined  by  the  memory.  For  lines  6,  7;  8  an^  9>  'a 
destination  code  is  sent  initially,  then  switched  to  the  control 
information  needed  by  the  destination  unit. 

Inbus  control  specifications  at  the  Unit -Exchange' interface 
are  the  same  as  those  at  the  Processor-Exchange  -interlace,  except 
that  only  those  signals  needed  by  the  receiving  unit  will  be  present, 
and  an  AU  Interrupt  signal  path  uses  line  3  to  go  to  the  AU. 

Outbus  control  specifications  at  the  Unit-Exchange  interface 
are  as  follows: 

line  Function 

1  Request 

Out  Information  Ready/ In  Information  Received 

3  Unit  Busy 

h  Multi-cycle  in  Progress 

5  Multi-cycle  Interrupt 

6  Bogus  Result 

7  Parity  Error 

8  Unit  Malfunction 

9  (Not  used) 

10        Unit  Attached 
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Each  unit  trends  out  only  the  information  needed.   Outbus 
control  specifications  at  the  Exchange-Processor  interface  are  exactly 
the  same  as  at  the  Unit-Exchange  interface,  except  that  line  9  now 
carries  an  IU  Calling  signal.   An  extra  10  bit  cable  goes  from  the 
IU  to  the  Exchange  Net,  and  carries  additional  control  signal;;: 

line  Function 

11  IU  Calling  Processor  0 

12  IU  Calling  Processor  1 

13  IU  Calling  Processor  2 

14  IU  Calling  Processor  3 

15  IU  Calling  Processor  k 

16  IU  Calling  Processor  5 

17  Encoded  Processor  Identification  (0) 

18  Encoded  Processor  Identification  (l) 

19  Encoded  Processor  Identification  (2) 

20  (Not  used) 

The  encoded  processor  identification  is  simply  000  for 
processor  0,  001  for  processor  1,  etc. 

With  the  above  specifications,  all  six  processors  can  change 
places  and  everything  will  still  work.   All  units  except  the  IU,  the 
AU' s  and  the  box  just  below  the  AU's  in  Figure  2,  can  interchange 
positions  as  long  as  the  processors  know  where  they  are  and  refer  to 
them  by  the  appropriate  destination  code.   The  destination  codes  listed 
in  Table  1  refer  to  the  terminals  to  which  the  units  shown  in  Figure  2 
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are  assigned.   Thi  two  AU' s  can  interchange  positions,  and  }  i'   a 
third  AU  is  found  to  be  desirable  in  the  future,  the  empty  termin; 
on  the  same  vertical  bus  can  be  used. 

3 . h      VarJjDUs  "Ub-nnits  in  th  '  E:.chnn.'-o  Met 

The  Exchange  Wet  can  be  divided  into  the  following  sub- 
units  (see  Figures  h   and  5): 

(1)  Six  Directors  --  Each  Director  examines  the  request 
and  destination  bits  from  one  processor.   If  there 
is  a  request,  it  stores  and  examines  the  destination 
code  and  presents  a  request  to  the  appropriate  Se- 
lector (see  below)  asking  for  the  unit  desired. 

(2)  Six  Selectors  --  Each  Selector  governs  the  use  of 
one  forward  vertical  bus.   It  looks  at  all  the  re- 
quests sent  in  by  the  different  Directors  and  checks 
for  a  number  of  conditions  for  every  request.   It  is 
only  when  all  these  conditions  are  met  that  a  request 
enters  a  priority  sequencing  and  transmission  is 
finally  granted  to  the  processor.   A  special  situa- 
tion occurs  for  processor-AU  communications.   In  this 
case,  the  particular  Selector  contains  an  AU  Selector 
which  selects  an  AU  while  checking  for  the  necessary 
conditions. 

(3)  Six  Select  Replies  --  Each  Select  Reply  is  associated 
with  a  processor,  and  scans  all  six  Selectors  for  the 
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U  =  UNIT 
SR  =  SELECT  REPLY 


NOTE  I:  THE  NUMBER  U  ACCOMPANIED  BY  2  DOTS 
BETWEEN  2  LINES  OR  BOXES  INDICATE 
M  SIMILAR  LINES  OR  BOXES  NOT 
SHOWN. 

NOTE  2:  AN  EXTRA  COMMUNICATION  PATH 
LEAD  FROM  LOCAL  EXCHANGE  (5) 
TO  SELECTOR  (5).  BUT  IS  NOT 
SHOWN  HERE  FOR  CLARITY  OF  THE 
GENERAL  PATTERN. 


SEE 
FIGURE  5 


FIGURE  4.  PROCESSOR  TO  UNIT  COMMUNICATION 
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selection  of  this  processor.   When  a  Selector  has  given 
the  processor  a  path,  the  Select  Reply  notifies  the 
processor  that  it  can  start  transmitting  data.   Since 
a  processor's  request  is  only  present  at  one  Selector, 
any  selection  signal  detected  by  the  Select  Reply  is 
from  that  Selector. 
(h)      One  Inward  Crosspoint  --  It  provides  a  six-by-six 
parallel  access  net  of  inbus  communications  from 
processors  to  Local  Exchanges.   The  actuation  of  the 
various  sets  of  gates  is  controlled  by  the  six 
Selectors . 

(5)  One  Outward  Crosspoint  --  It  provides  a  six-by-six 
parallel  access  net  of  outbus  communications  from 
Local  Exchanges  to  processors.   The  actuation  of  the 
various  sets  of  gates  is  controlled  by  the  six  Local 
Exchanges. 

(6)  Six  Local  Exchanges  --  For  any  information  coming  in 
from  the  processors,  a  Local  Exchange  records  the 
processor's  identification  and  sends  the  information 
to  the  appropriate  unit.   When  the  unit  wants  to  re- 
turn information  to  the  processor,  it  will  enable  the 
appropriate  set  of  gates  at  the  Outward  Crosspoint 
and  set  up  a  path  between  the  unit  and  the  processor. 
When  two  or  more  units  from  the  same  group  desire 
transmission,  the  priority  sequencing  in  the  Local 
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Exchange  will  decide  on  which  goes  first.   To  put  it 
in  another  way,  one  Local  Exchange  governs  the  use  of 
one  return  vertical  bus. 
(7)   Six  Bypasses  for  IU  Calling  Signals  --  The  six  IU 
Calling  lines  bypass  the  Local  Exchange  and  the 
Outward  Crosspoint  and  feed  directly  the  corresponding 
IU  Calling  lines  leading  into  the  processors  at  the 
Exchange-Processor  interface. 

3«5  Routing  of  Inbus  Traffic 

When  a  processor  wishes  to  communicate  with  a  unit,  it  puts 
up  a  request  signal  and  holds  it  until  a  path  is  closed  and  the 
processor  has  completed  transmitting  data.   At  the  time  a  processor 
puts  up  its  request,  it  must  also  present  to  the  Exchange  Net  a  four- 
bit  Destination  Code  that  specifies  the  unit  it  wants   to  reach. 
This  destination  code  only  has  to  last  long  enough  to  set  the  flip- 
flops  in  the  Director.,   Then  the  lines  can  carry  control  information 
to  be  sent  to  the  destination  unit.   The  Director  decodes  the  destina- 
tion and  sends  a  request  to  the  appropriate  Selector.   At  the  Selector, 
the  unit  requested  is  checked  for  conditions  like  "unit  attached", 
"no  malfunction"  and  "unit  not  busy".   When  all  conditions  are 
satisfied,  the  request  enters  a  round-robin  priority  sequencing  and 
will  finally  be  able  to  use  the  vertical  bus.   Thereupon,  the  processor 
is  notified  of  the  selection  by  the  Select  Reply,  and  an  "unit  activate" 
instruction  is  sent  by  the  Selector  to  the  Local  Exchange  to  actuate 


23 


the  appropriate  set  of  cable  driver  gates.   Thus  a  path  is  provided 
going  from  the  processor  to  the  desired  unit.   In  the  case  of  an  AU 
request,  when  the  request  is  sent  to  Selector  (5),  an  AU  is  chosen 
for  the  processor  while  all  the  necessary  conditions  are  being 
checked.   The  request  then  again  enters  a  priority  sequencing  and  an 
inbus  path  is  furnished  for  the  processor, 

3.6  Routing  of  Outbus  Traffic 

When  a  unit  wishes  to  communicate  with  a  processor,  it 
turns  on  its  request  signal  and  leaves  it  on  until  it  has  completed 
transmitting  data.   The  request  enters  directly  a  priority  sequencing 
and  the  unit  will  finally  be  allowed  to  use  the  return  vertical  bus. 
The  Local  Exchange  then  actuates  the  appropriate  set  of  cable  terminator 
gates  from  the  units  and  also,  except  for  the  IU,  the  set  of  gates  at 
the  Outward  Crosspoint  indicated  by  the  stored  processor  identifica- 
tion.  Thus  a  communication  path  is  furnished  leading  from  the  unit 
to  the  processor  that  has  last  communicated  with  it,  which  is  the 
one  wanted.   In  the  case  of  the  IU,  the  appropriate  cable  terminator 
gates  are  actuated,  but  the  set  of  gates  at  the  Outward  Crosspoint  is 
actuated  as  indicated  by  the  IU. 


2k 


h.      SUB-UNIT  DESCRIPTION 

k.l     Directors 

One  Director  is  provided  for  each  of  the  six  processors. 
Its  duty  is  to  examine  the  control  byte  from  its  processor  ana  check 
for  possible  requests.   Should  the  processor  desire  to  communicate 
with  a  unit  (i.e.  if  the  request  bit  from  the  processor  is  a  '1'), 
the  Director  decodes  the  Destination  Code  bits  in  the  control  byte 
and  causes  a  request  to  appear  on  an  appropriate  output  line.   This 
output  line  is  then  connected  to  the  correct  Selector  and  specifies 
the  desired  unit.   Since  the  Destination  Code  bits  time-share  the 
same  lines  as  the  Instruction  Variant  bits,  these  control  lines  can 
be  retained  for  only  a  short  period  of  time.  Accordingly  each 
Director  provides  its  own  flip-flops  to  store  the  code.   In  order 
not  to  waste  time  for  the  flip-flops  to  change  state,  the  Director 
initially  bypasses  these  flip-flops  and  decodes  the  Destination 
directly  from  the  control  lines.   However,  after  the  flip-flops  have 
been  set,  the  Director  uses  the  stored  Destination  Code,  thus  releasing 
the  control  lines  to  carry  other  information.   The  flow  diagram  for  a 
Director  is  given  in  Figure  6. 

h.2      Selectors 

Each  Selector  governs  the  inbus  communication  to  one  Local 
Exchange,  i.e  to  three  units.   Each  Selector  can  be  divided  into 
three  parts:   a  first  stage,  a  second  stage,  and  a  control.   Five  of 
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the  Selectors  are  identical,  while  the  sixth  Selector,  the  one 
governing  processor-AU  communications,  has  its  first  stage  replaced 
by  the  AU  Selector  described  in  the  next  section. 

For  each  of  the  first  five  Selectors,  there  are  three 
request  lines  coming  from  each  of  the  six  Directors  to  its  first 
stage  --  one  line  for  each  of  the  three  units  of  the  group.   Thus 
there  are  a  total  of  18  request  lines  entering  each  Selector  first 
stage.   The  first  stage  examines  all  18  lines  and  checks  for  each 
line  the  following: 

(1)  that  there  is  a  request 

(2)  that  the  unit  being  requested  is  connected  to  the 
Exchange  Net 

(3)  that  the  unit  is  not  busy 

(h)      that  there  is  no  unit  malfunction. 

It  is  only  when  all  of  the  above  conditions  are  met  that 
the  request  is  considered.   If  any  of  the  three  lines  from  the  same 
Director  passes  the  test,  a  request  signal,  representing  the  Director 
and  thus  its  associated  processor,  will  enter  the  second  stage  of  the 
Selector.   The  flow  diagram  for  the  first  stage  of  a  Selector,  with 
the  exception  of  the  AU  Selector,  is  given  in  Figure  7. 

The  second  stage  of  a  Selector,  provides  priority  sequencing 
and  the  final  provision  of  a  routing  path  for  the  processor.   Service 
by  the  Selector  is  on  a  round-robin  priority  basis.   The  scheme 
employed  is  this:   first,  there  is  a  wired-in  priority  scan  -- 
processor  0  has  the  highest  priority,  then  1,  then  2,  etc.   Whenever 
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more  than  one  request  is  presented  to  this  priority  scan,  the  processor 
with  the  highest  priority  will  be  selected.   Whenever  a  processor  is 
using  the  vertical  bus,  it  will,  when  a  priority  blocking  signal  is 
on,  erase  all  requests  with  a  higher  priority  than  itself  before  they 
enter  the  wired-in  priority  scan.   Thus,  if  2  is  the  current  user  of 
the  vertical  bus,  with  the  priority  blocking  signal  on,  requests  by 
0  and  1  are  blocked.   Any  request  for  2  while  it  is  using  the  bus 
is  of  course  invalid  and  is  erased  automatically.   So,  if  3  has  a  re- 
quest, it  will  be  honored  next.   If  not,  k   has  the  next  priority,  then 
5-   If  none  of  3*  ^  or  5  has  a  request,  the  priority  blocking  signal 
is  turned  off,  and  0  and  1  will  be  scanned,  which  now  have  the  highest 
priorities.   In  this  way,  a  round-robin  priority  scanning  effect  is 
acheived.   Should  5  be  using  the  bus,  no  requests  will  be  blocked 
even  if  the  priority  blocking  signal  is  on,  for  there  is  no  processor 
to  protect  with  a  priority  lower  than  that  of  5. 

When  a  processor  is  thus  selected,  its  request  signal  is 
then  gated  into  an  Interlock.   As  the  request  signal  enters  the 
Interlock,  several  things  happen.   First,  a  routing  path  is  provided 
between  the  processor  and  the  Local  Exchange  on  top  of  the  vertical 
bus.   Next,  the  processor  is  informed  of  its  being  granted  a  routing 
path.   Then,  the  Local  Exchange  is  informed  of  where  the  data  is 
coming  from,  and  where  it  is  going  to,  so  that  it  can  route  the  data 
to  the  correct  unit. 

While  a  processor  is  utilizing  a  vertical  bus,  its  status 
in  the  Interlock  cannot  be  changed,  so  no  processor  can  disturb  its 
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transmission.  As  soon  as  the  processor  releases  the  bus,  the  Inter- 
lock is  cleared,  and  it  is  ready  to  service  the  next  request.   The 
flow  diagram  for  the  second  stage  of  a  Selector  is  given  in  Figure  8. 

The  control  part  of  the  Selector  functions  mainly  as  the 
supervisor  for  the  second  stage.  It  examines  at  all  times  the  re- 
quest bit  being  sent  to  the  Local  Exchange  on  top  of  the  Selector, 
and  senses  for  completion  signals  given  by  users  of  the  vertical  bus, 
and  it  generates  all  the  signals  needed  to  operate  the  second  stage 
properly.  The  flow  diagram  for  the  control  of  a  Selector  is  given 
in  Figure  9* 

k.3     Arithmetic  Unit  Selector 

When  a  processor  asks  for  an  arithmetic  unit,  it  does  not 
specify  which  of  the  two  A U' s  it  wants.   When  the  Director  receives 
the  AU  Request  signal  from  its  processor,  it  forwards  the  request  to 
the  AU  Selector,  which  selects  an  AU  according  to  certain  criteria: 

(1)  If  the  processor  has  been  doing  a  multi-cycle  opera- 
tion, it  is  the  duty  of  the  AU  Selector  to  direct  the 
serie;;  of  data  words  to  the  same  AU. 

(2)  If  both  AU' s  are  free  from  multi-cycle  operations, 
choose  one  that  is  not  busy.   If  both  are  busy,  wait, 
then  choose  the  one  that  finishes  first. 

(3)  If  one  AU  is  doing  a  multi-cycle  operation  (not 

for  the  requesting  processor),  and  the  other  is  not, 
choose  the  one  that  is  not  doing  the  multi-cycle 
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operation.   If  it  is  busy,  wait,  then  ask  for  it. 
Do  not  interrupt  the  other  one. 
(k)      If  both  AU' s  are  doing  multi-cycle  operations  and 

none  is  for  the  the  requesting  processor,  interrupt 
one  of  the  AU's.   When  the  AU  is  clear,  ask  for  it. 
Other  considerations  pertinent  to  the  design  of  the  AU 
Selector  are: 

(1)  As  soon  as  an  AU  receives  an  Instruction  from  a  TP 
and  recognizes  that  it  is  a  multi-cycle  operaton,  it 
turns  on  its  "multi-cycle  in  progress"  signal  and 
keeps  it  on  until  it  has  received  the  last  instruc- 
tion of  the  multi-cycle  operation. 

(2)  During  a  multi-cycle  operation,  the  AU  talks  back  to 
the  instructing  TP  each  time  it  arrives  at  a  partial 
result  and  before  the  TP  sends  in  the  next  operand. 

(3)  As  will  be  described  under  the  section  on  Local 
Exchanges,  there  are  registers  in  the  AU-associated 
Local  Exchange  that  remember  the  identifications  of 
the  processors  that  have  last  communicated  with  the 
AU's. 

(h)      The  AU  turns  on  its  busy  signal  when  it  receives  an 

instruction  and  turns  it  off  when  it  is  free  from  its 
present  duties.   For  a  multi-cycle  operation,  the  AU 
keeps  its  busy  signal  on  until  the  whole  operation 
is  completed  and  the  final  result  is  sent  back  to 
the  TP. 
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('_))     When  an  AU  receives  an  interrupt  signal,  it  immediately 
turns  off  its  "multi-cycle  in  progress"  signal.  As 
soon  as  it  finishes  its  current  operation,  it 
sends  back  the  partial  result  to  the  TP  and  turns  off 
the  AU  Busy  signal. 
By  examining  the  "multi-cycle  in  progress"  bit  from  each  of 
the  AU's,  the  AU  Selector  knows  if  an  arithmetic  unit  is  doing  a  multi- 
cycle operation.   Using  this  signal  to  gate  the  appropriate  processor 
identification  from  the  Local  Exchange,  the  AU  Selector  obtains  for  its 
use  the  identification  of  the  processor  that  is  doing  a  multi-cycle 
operation  in  any  AU.   When  a  Director  forwards  a  request  to  the  AU 
Selector,  the  AU  Selector  first  checks  to  see  if  the  processor  has 
been  doing  any  multi-cycle  operations.   If  the  processor  matches 
either  of  the  two  AU  multi-cycle  user  identifications,  that  same  AU 
is  chosen  for  the  processor.   If  this  test  yields  a  negative  result, 
an  AU  interrupt  occurs  should  both  AU' s  be  doing  multi-cycle  opera- 
tions.  If  not  (or  after  an  interrupt  has  occurred  and  the  AU  is 
cleared),  each  AU  is  examined  for  the  following  conditions: 

(1)  that  the  AU  is  not  busy 

(2)  that  there  is  no  multi-cycle  operation  going  on  at 
the  AU  or  that  the  AU  is  cleared  up  after  an  interrupt 

(3)  that  the  AU  is  attached  to  the  Exchange  Net 
(k)      that  there  is  no  malfunction  at  the  AU 

Since  the  AU  Busy  signal  covers  the  whole  range  of  the 
Multi-cycle  in  Progress  signal,  condition  2  is  met  whenever  condition 
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1  is  fulfilled,  and  only  conditions  1,  3  and  k   are  actually  checked. 
If  the  first  AU  passes  the  test,  it  is  chosen  by  the  AU  Selector.   If 
not,  the  second  AU  is  chosen  if  it  passes  the  test.   If  both  fail  the 
test,  all  that  the  AU  Selector  will  do  is  wait  until  one  is  cleared 
with  the  checking.   Should  one  processor  get  an  AU  that  is  also  being 
requested  by  a  second  processor,  the  AU  Selector  accordingly  will 
request  the  other  AU  for  this  second  processor,  or  wait*   The  flow 
diagram  for  the  AU  Selector  is  given  in  Figure  10. 

h.k     Select  Replies 

A  Select  Reply  is  essentially  a  relay  service  that  transmits 
the  Processor  Select  signal  from  any  Selector  to  a  given  processor. 
It  scans  all  six  Selectors  to  see  if  any  of  them  has  granted  a  vertical 
bus  to  the  processor  that  it  is  working  for.   If  so,  it  informs  the 
processor  that  its  request  has  been  granted  and  can  start  transmitting 
data. 

4.5  Inward  Crosspoint 

The  Inward  Crosspoint  is  a  six-by-six  communication  net  of 
Inbus  traffic .   It  is  capable  of  handling  the  parallel  flow  of  data 
from  all  six  processors  to  all  six  Local  Exchanges  simultaneously, 
provided  that  the  rule  of  "one  processor  to  one  Local  Exchange"  is 
followed.   The  six  sets  of  gates  on  a  forward  vertical  bus  are  contr611ed 
by  one  Selector.   Because  of  the  way  a  Selector  functions  (as  ex- 
plained in  the  section  on  Selectors),  the  regulation  that  only  one 
processor  can  talk  to  a  Local  Exchange  at  a  time  is  always  obeyed. 
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FIGURE  10  FLOW  DIAGRAM  FOR  THE  AU  SELECTOR 
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Referring  to  Figure  11,  the  Inward  Crosspoint  consists  of 
the  six-by-six  crossbar  that  enables  the  flow  of  inbus  traffic. 
Suppose  TP  (0)  is  to  send  data  to  a  memory  unit  connected  to  LE  (2), 
the  five  bytes  of  inbus  information  will  appear  at  all  six  inter- 
sections that  the  horizontal  bus  makes  with  the  six  vertical  buses. 
The  Director  forwards  TP  (0)'s  request  to  Selector  (2),  and  when  the 
request  is  granted,  the  TP  (0)  — -  LE  (2)  junction  is  actuated  and  data 
from  TP  (0)  is  transmitted  to  LE  (2). 

k.6      Outward  Crosspoint 

The  Outward  Crosspoint  is  exactly  like  the  Inward  Crosspoint 
(see  Figure  12).   It  also  consists  of  a  six-by-six  crossbar  but  this 
one  provides  a  parallel  access  net  of  Outbus  traffic.   The  six  sets 
of  gates  on  a  return  vertical  bus  are  controlled  by  one  Local  Excange. 
Since  units  only  send  results  to  the  processor  which  last  accessed 
them  (with  the  exception  of  the  IU),  and  since  a  processor  will  never 
access  a  second  unit  unless  it  has  finished  with  the  first,  no  two 
units  will  ever  access  the  same  processor  at  the  same  time.   The 
Interrupt  Unit  knows  the  conditions  of  all  processors,  and  will 
forward  an  interrupt  control  word  only  when  the  processor  is  cleared. 
Therefore,  the  regulation  that  only  one  Local  Exchange  can  access  a 
given  processor  at  any  one  time  will  not  be  violated.   The  flow  of 
traffic  through  the  Outward  Crosspoint  is  completely  independent  of 
the  flow  of  traffic  through  the  Inward  Crosspoint  and  vice  versa. 
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LE  =  LOCAL  EXCHANGE 
TP  =  TAX  I CR I N I C  PROCESSOR 
I  OP  =  INPUT/OUTPUT    PROCESSOR 
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LE  =  LOCAL  EXCHANGE 
TP  =  TAXI CRI NIC  PROCESSOR 
I  OP  =  INPUT/OUTPUT  PROCESSOR 
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k.J     Local  Exchanges 

One  Local  Exchange  governs  the  communications  to  and  from 
three  units.   When  a  Selector  grants  a  processor  a  routing  path,  it 
at  the  same  time  notifies  its  corresponding  Local  Exchange  of  which 
processor  is  sending  in  data,  and  what  unit  this  data  is  to  be  sent 
to.   The  Local  Exchange  then  actuates  an  approprate  set  of  cable 
driver  gates  and  routes  all  data  to  the  desired  unit.   While  it  is 
doing  this,  it  also  stores  the  processor's  identification  in  a  register, 
so  that  when  the  unit  wants  to  return  information  to  its  last  contact, 
the  Local  Exchange  is  able  to  direct  it  to  the  correct  processor .  An 
exception  occurs  in  the  case  of  the  IU.  The  processor's  identifica- 
tion is  not  stored,  and  the  IU  specifies  the  processor  each  time  it 
wants  communication. 

When  a  unit  wants  to  communicate  with  a  processor,  it  sends 
a  request  to  the  Local  Exchange.  A  Local  Selector  inside  the  Local 
Exchange  selects  one  of  the  three  possible  requests.   This  Local 
Selector  is  just  like  a  Selector  but  without  the  first  stage  and  has 
only  three  request  lines.  When  the  unit's  request  is  granted  by  the 
Local  Selector,  the  Local  Exchange  actuates  the  cable  terminator  gates 
from  the  unit  and  fetches  the  processor's  identification  from  the 
appropriate  register  to  actuate  the  correct  set  of  gates  at  the 
Outward  Crosspoint.   This  provides  a  path  from  the  unit  to  the 
processor.  As  mentioned  above,  the  IU  specifies  directly  the  pro- 
cessor it  wants  to  reach,  and  Local  Exchange  (k)   uses  this  information 
to  actuate  the  Outward  Crosspoint  gates.   Figure  13  and  Ik   show  respec- 
tively the  flow  diagrams  of  the  Inbus  and  Outbus  portions  of  the  Local 
Exchange . 
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FIGURE  13.  FLOW  DIAGRAM  FOR  THE  INBUS  PORTION  OF  A  LOCAL  EXCHANGE 
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U.8   Bypasses  for  Interrupt  Unit  Calling  Signals 

After  the  six  IU  Calling  lines  enter  the  Exchange  Net,  they 
are  taken  directly  to  the  Exchange-Processor  interface  and  fed  to  the 
corresponding  IU  Calling  lines  on  the  outbus .   "IU  Calling  processor 
0"  is  dot-OR-ed  with  outbus  control  bit  9  of  processor  0,  "IU  Calling 
processor  1"  is  dot-OR-ed  with  outbus  control  bit  9  of  processor  1, 
etc.   Since  the  units  do  not  use  bit  9  of  the  outbus  control  byte, 
they  cause  this  hit  to  be  '0'  at  both  Unit-Exchange  and  Exchange- 
Processor  interfaces,  or  the  higher  voltage  level  with  our  negative 
logic  convention.   Therefore,  if  the  IU  Calling  signal  is  '1',  the 
dot-OR  will  produce  a  logical  '1'  and  a  ' 1'  is  received  by  the 
processor.   If  the  IU  Calling  signal  is  a  '0',  the  dot-OR  will  produce 
a  logical  '0'  and  a  '0'  is  received  by  the  processor. 
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5 .   SUMMARY 

A  communication  net  of  modular  processors  for  the  multi- 
processor computer  Illiac  III  is  modeled  in  this  paper,   The  model 
contains  design  features  which  (l)  allow  the  computer  to  ope..,  ;e  with 
a  variable  number  of  modules,  (2)  allow  modules  within  a  group  to  use 
alternate  terminals,  so  that  when  one  terminal  fails  to  function 
properly,  the  module  can  be  reattached  to  another  one,  (3)  eliminate 
the  hazard  of  the  parallel  access  of  the  same  module  by  two  others 
where  this  is  not  allowed.  All  the  requirements  imposed  on  the 
communication  net  by  the  surrounding  units  are  met,  and  an  appropriate 
interface  convention  has  been  established.   The  system  design  presented 
in  this  paper  has  been  implemented  by  a  logical  design  reported  in  the 
Illiac  III  Manual  now  in  preparation . 
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